Upgrade OpenGL backend to ES 3.0 #995

mwilsnd · 2023-04-10T17:43:30Z

This PR upgrades the shaders and GL contexts of all platforms (minus darwin) to OpenGL ES 3.0. A large chunk of changes here are in the shaders & generated shader code (mbgl/shaders/, shaders/) where the following changes were made:

#version 300 es support
- attribute -> layout (location = n) in
- varying -> in/out
- texture2D -> texture
- gl_FragColor -> explicit output color
- The shader code generator was upgraded to emit vertex attribute layout qualifiers

All instances of MBGL_USE_GLES2 have been removed, with older pathways for ES 1 stuff deleted. All context creation methods have been updated to request ES 3 or ES 3 compatible (windows) contexts, save for macOS's CoreGL backend (support here is pending, may be done via ANGLE in the future).

include/mbgl/shaders/gl/circle.hpp

sjg-wdw

Nice to see the 3.0 syntax in the shaders.

TimSylvester

Merges and works fine with my pending changes 👍

ntadej

Let me open a pending review before I have the chance to test Qt, should be soon.

ntadej · 2023-04-11T19:38:45Z

No, Qt does not work anymore on macOS. I guess then Node.js would also be broken. What's the motivation to move to GLES 3.0?

mwilsnd · 2023-04-11T19:47:16Z

No, Qt does not work anymore on macOS. I guess then Node.js would also be broken. What's the motivation to move to GLES 3.0?

Any platform built for macOS is going to fail with this PR - Qt, node, render-test-runner and so on. Moving to ES 3 gives us a few important features, the two big ones are uniform buffer objects (UBOs) and Vertex array objects (VAOs). Using these, we can significantly simplify the existing GL rendering code and bring it more inline with what we anticipate the coming Metal backend will look like. ES 3 also brings increased minimums on certain implementation limits like number of vertex attributes, which increases to 16 with ES 3, removing one of the reasons for keeping the existing shader permutation system around.

The current plan then, is merge this PR and shut off macOS on CI. Existing users on macOS will have to remain on the last stable release until the Metal platform is ready later this year. Once Metal is ready, the darwin platform (and all dependents like glfw, Qt, node) can be moved to the new native backend and revived.

The other option is to bridge the gap with ANGLE on mac, but this is extra temporary work until Metal is implemented. This option should only be entertained if there is significant enough demand for it.

ntadej · 2023-04-11T20:12:46Z

OK, then we should cut releases before this is merged. Also this motivates more repo split as I effectively can't develop anymore mac being my primary platform.

To be safe, we should also manually test other platforms I guess (Windows is another platform I'm worried about).

Let's discuss this tomorrow how to best proceed (and if you're willing to eventually help supporting Metal in Qt 😉).

mwilsnd · 2023-04-11T20:22:17Z

OK, then we should cut releases before this is merged. Also this motivates more repo split as I effectively can't develop anymore mac being my primary platform.

To be safe, we should also manually test other platforms I guess (Windows is another platform I'm worried about).

Let's discuss this tomorrow how to best proceed (and if you're willing to eventually help supporting Metal in Qt 😉).

Windows works fine - Pictured is the GLFW platform running on Windows. Test runners also "work" (The mapbox filesystem impl doesn't support windows dir separators and explodes trying to find test metrics).

sjg-wdw · 2023-04-11T20:39:48Z

Does Qt not support Metal?

ntadej · 2023-04-11T20:44:35Z

Qt6 fully supports Metal (Qt5 I'd have to check). The problem is I barely know bits of OpenGL so I'll need help gluing things together. Unfortunately they're missing some Metal examples that they have for Vulcan and D3D11. It will probably be minimal work for an expert though.

TimSylvester · 2023-04-12T15:13:50Z

It looks like there are new definitions that should be added to mbgl/gl/defines.hpp, e.g., GL_UNSIGNED_INT_VEC2 from https://docs.gl/es3/glGetActiveAttrib.

mwilsnd · 2023-04-12T15:19:42Z

It looks like there are new definitions that should be added to mbgl/gl/defines.hpp, e.g., GL_UNSIGNED_INT_VEC2 from https://docs.gl/es3/glGetActiveAttrib.

Being addressed in #1007

louwers · 2023-04-13T16:05:58Z

@ntadej OK to merge?

ntadej · 2023-04-13T17:09:16Z

First the CI should be green (so failing tests disabled) 😊

We may want to tag some releases before we merge this though. I'll be travelling tomorrow but I will be available on the weekend. I think @acalcutt also wanted to tag a release before this is merged.

Can this wait a few days? I can also merge it when green when I'm ready (and the CI is green).

louwers · 2023-04-13T19:12:17Z

Can this wait a few days?

Not really I think. This is a blocker for quite a lot of other things. (cc @sjg-wdw)

If you want to make a release, it's possible to do it from the opengl-2 branch?

acalcutt · 2023-04-13T19:40:23Z

I tried to do the last node release i wanted to do but it failed to bump the version. wonder if that node-pregyp-githib secret expired @birkskyum ?

ntadej · 2023-04-13T19:41:03Z

OK, I can also tag from some older commit the way how the release is set-up (only the changelog may not be OK). So from my side you're OK to proceed.

birkskyum · 2023-04-13T19:49:52Z

@acalcutt it hasn't expired

birkskyum · 2023-04-13T20:14:21Z

@acalcutt , unless github has made some updates to their personal token settings (.. they do work on fine-grained repo-scoped personal tokens), my best guess is that it's caused by the branch protection in this repo. Currently the branch protection setting for main has enabled "Do not allow bypassing the above settings", which is good practice, but if I don't remember wrong the node deployment rely on auto-pushing a release commit directly to main effectively bypassing the PR requirement.

acalcutt · 2023-04-13T20:18:45Z

Before we were using that secret to get passed the branch protection on the version bump, since we checked out with that user also. must not be working anymore. ill have to look into it and see if i have the same issue on my fork. not sure the workflow is set up to build a release for a branch..hmm.

probably wont be able to look until later

louwers · 2023-04-13T20:24:40Z

@ntadej We had a discussion about keeping macOS CI running on main.

In the end @mwilsnd, @alexcristici and me decided against it. We can apply some tricks to make the project build (but not run) for macOS on main, but this creates the false expectation that macOS will work on main. As we decided during the TSC meeting, until the Metal migration is complete macOS will be broken on main and only functioning on the opengl-2 branch. Since macOS support is one of the north stars of the design proposal and thus one of the deliverables of the team, I think we can be confident that the team will get to bringing back macOS support in due time.

Hope you can agree with this reasoning. Maybe you can approve the PR explicitly, otherwise I can't merge (without dismissing your review).

birkskyum · 2023-04-13T20:27:00Z

@acalcutt , the token used on the fork, can you see what permissions it has? The one used here has only public_repo, repo_deployment, and I actually wouldn't expect that to work because they are read-only... I wonder if they changed this list of permissions. Would expect something like write:repo_hook to be needed

"OK to proceed."

…awable-1 * commit 'f100a4c5d6449307547c2da850db04b12c6a3a95': Minor reorg of defines. Removed more commented code. Removed commented code. Upgrade OpenGL backend to ES 3.0 (maplibre#995) Run macOS workflows only on `opengl-2` branch (maplibre#1012) Roll back linux metrics submit along with macos metrics. (maplibre#1009)

tdcosta100 · 2023-04-26T05:44:06Z

Hi guys. I know it's been a while since this PR was merged, but platform/windows/FindOSMesa.cmake doesn't fit these changes, because Mesa3D doesn't support GLES 3.0, only GLES 1.1 and GLES 2.0.

I'm planning to submit a PR to fix this, please validate my plans:

Drop Mesa3D support in main branch, since the minimum supported version is 3.0
Move/keep Mesa3D support to opengl-2 branch

Is this the correct way of doing this change?

Thank you in advance.

acalcutt · 2023-04-26T11:29:45Z

@tdcosta100 , does that depend on the driver used? from this page it seems like mesa does support ES 3.0

https://mesamatrix.net/

tdcosta100 · 2023-04-26T20:52:06Z

Nice, I will try to test it creating a GLES 3 surface. If it works, I will just adjust it (Mesa3D doesn't have a libGLESv3.dll file).

tdcosta100 · 2023-04-26T22:27:40Z

Yeah, it works, but interesting is: the DLL name is libGLESv2.dll, even it holds all the ES 3.0 functions. I will submit a new PR correcting this.

mwilsnd requested review from alexcristici, TimSylvester and sjg-wdw April 10, 2023 17:43

mwilsnd self-assigned this Apr 10, 2023

mwilsnd changed the title ~~Gles3~~ Upgrade OpenGL backend to ES 3.0 Apr 10, 2023

alexcristici reviewed Apr 10, 2023

View reviewed changes

include/mbgl/shaders/gl/circle.hpp Show resolved Hide resolved

sjg-wdw approved these changes Apr 10, 2023

View reviewed changes

TimSylvester approved these changes Apr 10, 2023

View reviewed changes

alexcristici approved these changes Apr 10, 2023

View reviewed changes

mwilsnd marked this pull request as ready for review April 11, 2023 17:13

mwilsnd requested review from louwers and ovivoda April 11, 2023 17:14

ntadej previously requested changes Apr 11, 2023

View reviewed changes

louwers mentioned this pull request Apr 11, 2023

Run iOS C++ unit tests on CI #1001

Closed

alexcristici mentioned this pull request Apr 12, 2023

API links for OpenGL ES 3.0 #1007

Merged

louwers mentioned this pull request Apr 13, 2023

Run macOS workflows only on opengl-2 branch #1012

Merged

louwers approved these changes Apr 13, 2023

View reviewed changes

louwers enabled auto-merge (squash) April 13, 2023 16:06

louwers requested a review from ntadej April 13, 2023 16:08

mwilsnd added 10 commits April 13, 2023 21:32

Upgrade shaders to #version 300 es

47c8c77

generate_shader_code.js emits layout qualifiers

29563df

Shaders specify fragColor output and #version 300

7c57af5

Line width is not deprecated!

6a3db45

ES 3 shader conformance

4dc9204

Half floats are spec in ES 3

526bfeb

ES3-ify Windows platform

549c9dc

ES3-ify Android platform

d0bb4b3

ES3-ify iOS & Darwin (EAGL)

9b3e5f3

Update CMake & remove ES1 code from glfw platform

71028bc

louwers force-pushed the gles3 branch from 153087f to 71028bc Compare April 13, 2023 19:32

Disable macOS-ci on main

bc288d5

louwers merged commit 4828cc7 into maplibre:main Apr 13, 2023

mwilsnd deleted the gles3 branch April 14, 2023 17:19

tdcosta100 mentioned this pull request Apr 27, 2023

Revert GLES3 changes for OSMesa in Windows #1079

Merged

vng mentioned this pull request Jun 10, 2023

[macos] Tried to make it works with OpenGL >= 3.3 #1235

Draft

acalcutt mentioned this pull request Jan 16, 2024

cmake: enable MLN_WITH_EGL and OPENGL_USE_GLES for Wayland #2022

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upgrade OpenGL backend to ES 3.0 #995

Upgrade OpenGL backend to ES 3.0 #995

mwilsnd commented Apr 10, 2023 •

edited

Loading

sjg-wdw left a comment

TimSylvester left a comment

ntadej left a comment

ntadej commented Apr 11, 2023

mwilsnd commented Apr 11, 2023

ntadej commented Apr 11, 2023

mwilsnd commented Apr 11, 2023

sjg-wdw commented Apr 11, 2023

ntadej commented Apr 11, 2023

TimSylvester commented Apr 12, 2023

mwilsnd commented Apr 12, 2023

louwers commented Apr 13, 2023

ntadej commented Apr 13, 2023

louwers commented Apr 13, 2023

acalcutt commented Apr 13, 2023

ntadej commented Apr 13, 2023

birkskyum commented Apr 13, 2023 •

edited

Loading

birkskyum commented Apr 13, 2023 •

edited

Loading

acalcutt commented Apr 13, 2023

louwers commented Apr 13, 2023 •

edited

Loading

birkskyum commented Apr 13, 2023

tdcosta100 commented Apr 26, 2023 •

edited

Loading

acalcutt commented Apr 26, 2023

tdcosta100 commented Apr 26, 2023

tdcosta100 commented Apr 26, 2023

Upgrade OpenGL backend to ES 3.0 #995

Upgrade OpenGL backend to ES 3.0 #995

Conversation

mwilsnd commented Apr 10, 2023 • edited Loading

sjg-wdw left a comment

Choose a reason for hiding this comment

TimSylvester left a comment

Choose a reason for hiding this comment

ntadej left a comment

Choose a reason for hiding this comment

ntadej commented Apr 11, 2023

mwilsnd commented Apr 11, 2023

ntadej commented Apr 11, 2023

mwilsnd commented Apr 11, 2023

sjg-wdw commented Apr 11, 2023

ntadej commented Apr 11, 2023

TimSylvester commented Apr 12, 2023

mwilsnd commented Apr 12, 2023

louwers commented Apr 13, 2023

ntadej commented Apr 13, 2023

louwers commented Apr 13, 2023

acalcutt commented Apr 13, 2023

ntadej commented Apr 13, 2023

birkskyum commented Apr 13, 2023 • edited Loading

birkskyum commented Apr 13, 2023 • edited Loading

acalcutt commented Apr 13, 2023

louwers commented Apr 13, 2023 • edited Loading

birkskyum commented Apr 13, 2023

tdcosta100 commented Apr 26, 2023 • edited Loading

acalcutt commented Apr 26, 2023

tdcosta100 commented Apr 26, 2023

tdcosta100 commented Apr 26, 2023

mwilsnd commented Apr 10, 2023 •

edited

Loading

birkskyum commented Apr 13, 2023 •

edited

Loading

birkskyum commented Apr 13, 2023 •

edited

Loading

louwers commented Apr 13, 2023 •

edited

Loading

tdcosta100 commented Apr 26, 2023 •

edited

Loading